© Marek Cierpiał-Wolan, Galya Stateva. Artykuł udostępniony na licencji CC BY-SA 4.0
In view of many dynamic changes taking place in the modern world due to the pandemic, the migration crisis, armed conflicts, etc., it is a huge challenge for official statistics to provide good-quality information, which should be available almost in real time. In this context, integration of data from multiple sources, in particular big data, is a prerequisite.
The aim of the article is to characterise and evaluate the following selected methods of data integration in tourism statistics: Natural Language Processing (NLP), machine learning algorithm, i.e. K-Nearest Neighbours (K-NN) using TF-IDF and N-gram techniques, and Fuzzy Matching, belonging to probabilistic methods.
In tourism surveys, data acquired using web scraping deserve special attention. For this reason, the analysed methods were used to combine data from booking portals (Booking.com, Hotels.com and Airbnb.com) with a tourism survey frame. An attempt was also made to answer the question of how the data obtained from web scraping of tourism portals improved the quality of the frame. The study showed that Fuzzy Matching based on the Levenshtein algorithm combined with Vincenty’s formula was the most effective among all tested methods. In addition, as a result of data integration, it was possible to significantly improve the quality of the tourism survey frame in 2023 (an increase in the number of new accommodation establishments in Poland by 1.1% and in Bulgaria by 1.4%).
data linkage methods, tourism survey frame, web scraping
C1, C81, Z32
Asher, J., Resnick, D., Brite, J., Brackbill, R., & Cone, J. (2020). An Introduction to Probabilistic Record Linkage with a Focus on Linkage Processing for WTC Registries. International Journal of Environmental Research and Public Health, 17(18), 1–16. https://doi.org/10.3390/ijerph17186937.
Christen, P. (2012). Data Matching. Concepts and Techniques for Record Linkage, Entity Resolution, and Duplicate Detection. Springer. https://doi.org/10.1007/978-3-642-31164-2.
Cierpiał-Wolan, M., Truszyńska, A., Szlachta, P., Wnuk, Z., Sawicki, K., Oprych-Franków, D., Data, M., Ulma-Ciupak, B., Giełbaga, E., Wieczorek, G., Gumiński, M., & Mordan, P. (2022). Feasibility project on digitalisation issues in national accounts.
Cierpiał-Wolan, M., & WPJ Team. (2020). Innovative Tourism Statistics Deliverable J2: Interim technical report showing the preliminary results and a general description of the methods used. Eurostat, ESSnet Big Data II. https://ec.europa.eu/eurostat/cros/sites/default/files/WPJ_Deliverable_J2_Interim_technical_report_showing_the_preliminary_results_and_a_general_description_of_the_methods_used_2020_01_07.pdf.
Cierpiał-Wolan, M., Zadorożny, Ł., Szlachta, P., Matuła, T., Data, M., & Gawełko, J. (2023). Report on granular deduplication methods – Deliverable 2.2.
Daas, P., Ossen, S., Vis-Visschers, R., & Arends-Tóth, J. (2009). Checklist for the Quality evaluation of Administrative Data Sources (CBS Discussion Paper No. 09042). https://ec.europa.eu/eurostat/documents/64157/4374310/45-Checklist-quality-evaluation-administrative-data-sources-2009.pdf/24ffb3dd-5509-4f7e-9683-4477be82ee60.
European Commission. (n.d. a). Project Overview. Retrieved July 8, 2023, from https://cros-legacy.ec.europa.eu/content/project-overview_en.
European Commission. (n.d. b). WPJ Innovative tourism statistics. Retrieved July 8, 2023, from https://cros-legacy.ec.europa.eu/content/WPJ_Innovative_tourism_statistics.
Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874. https://doi.org/10.1016/j.patrec.2005.10.010.
Maślankowski, J. (2015). Analiza jakości danych pozyskiwanych ze stron internetowych z wykorzystaniem rozwiązań Big Data. Roczniki Kolegium Analiz Ekonomicznych SGH, (38), 167–177. https://rocznikikae.sgh.waw.pl/p/roczniki_kae_z38_11.pdf.
Peirce, C. S. (1884). The Numerical Measure of the Success of Predictions. Science, 4(93), 453–454. https://doi.org/10.1126/science.ns-4.93.453-a.
Powers, D. M. W. (2011). Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation. Journal of Machine Learning Technologies, 2(1), 37–63. https://bioinfopublication.org/pages/article.php?id=BIA0001114.
Quinlan, R. (1983). Learning efficient classification procedures. In R. S. Michalski, J. G. Carbonell & T. M. Mitchell (Eds.), Machine Learning. An Artificial Intelligence Approach (pp. 463–482). Springer-Verlag. https://doi.org/10.1007/978-3-662-12405-5.
United Nations Department of Economic and Social Affairs Statistics Division. (2015). Classification of Types of Big Data. https://unstats.un.org/unsd/classifications/expertgroup/egm2015/ac289-26.PDF.
United Nations Economic Commission for Europe. (n.d.). Unece Statswiki. Retrieved July 8, 2023, from https://statswiki.unece.org/display/bigdata/Classification+of+Types+of+Big+Data.
Youden, W. J. (1950). Index for rating diagnostic tests. Cancer, 3(1), 32–35. https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3.
